Summary

This PR is the consolidated closeout branch for the long-running PairOfCleats roadmap work. It turns the branch from a collection of scattered roadmap/spec/test/performance efforts into a single current implementation surface with durable documentation, guarded contracts, cleaned shared-module ownership, explicit duplicate-code policy, production-readiness gates, and current performance evidence.

The branch has been pushed to origin/NEON_TIDE. Live diff shape against main at update time:

• 4315 files changed, 370136 insertions(+), 140963 deletions(-).
• Main changed areas: tests (2681 files), src (897), tools (370), docs (302), plus editor integration, workflow, CLI, and TUI surfaces.
• Current head: 09dd102bd perf: clean sweet16 performance signals.
• The PR is open and non-draft. GitHub currently reports the merge state as DIRTY, so the remaining integration concern is conflict resolution against main, not unpushed local work.

What changed

• Roadmap/spec consolidation: collapsed the branch's scattered roadmap, implementation-plan, backlog, readiness, and stale approval-lock material into docs/roadmap.md plus archived historical docs, so active work is tracked from one current status surface backed by technical evidence.

Unified Syntax Representation (USR)

• USR is PairOfCleats' Unified Syntax Representation: the canonical syntax/semantic model used by the indexing pipeline, language registry, framework extraction, graph surfaces, risk analysis, and query/retrieval consumers.
• The problem it solves is that PairOfCleats supports many languages and framework profiles, but raw parser outputs are not interchangeable. Native parsers, tree-sitter adapters, heuristics, generated framework/compiler segments, and partial-support fallbacks all expose different node kinds, coordinate spaces, identity behavior, and degradation modes. USR gives those paths one deterministic contract before downstream graph/query/risk code consumes them.
• The model standardizes document, segment, syntax-node, symbol, edge, range, diagnostic, capability, and evidence semantics across the supported language registry and framework overlays. That includes stable IDs, deterministic ordering, repo-relative POSIX paths, UTF-16 coordinate rules, explicit null/empty/omitted-field behavior, finite numeric normalization, partial-support signaling, and preserved raw parser/compiler kinds alongside normalized kinds.
• Framework profiles are first-class overlays rather than ad hoc exceptions. React, Vue, Next.js, Nuxt, Svelte/SvelteKit, Angular, and Astro behavior is represented through explicit segmentation, binding, route, hydration, bridge, and edge-case matrices instead of hidden per-parser assumptions.
• The implementation moves USR from prose intent to enforceable machinery:
  • docs/specs/unified-syntax-representation.md remains the umbrella contract for representation semantics.
  • docs/specs/usr-core-*.md and docs/specs/usr/** define the focused language, framework, quality, governance, rollout, security, diagnostics, and evidence contracts.
  • docs/schemas/usr/** defines machine-readable evidence/report schemas.
  • src/contracts/schemas/usr.js and src/contracts/validators/usr*.js enforce schema shape, diagnostics, reason codes, matrix registries, cross-registry invariants, and report artifact IDs.
  • tools/usr/generate-usr-matrix-baselines*.mjs owns generated registry/matrix baselines so language/framework capability expectations stay deterministic.
  • tools/usr/conformance-surface.js, tests/usr-full-conformance/**, conformance shards, docs guards, and runner lanes provide current branch proof instead of relying on stale checklist approval language.
• The branch added and refreshed language/framework specs and machine-readable matrices for the current supported surface, including required capabilities, required node/edge kinds, parser preferences, fallback chains, framework applicability, embedded-language bridges, version/backcompat policy, quality gates, waiver/ownership/escalation policy, evidence freshness, and lane policy.
• The branch also reconciled the release/security/risk docs to the current schema-backed evidence artifacts:
  • usr-validation-report.json
  • usr-threat-model-coverage-report.json
  • usr-failure-injection-report.json
• Guardrails now fail when a release plan references a schema-backed USR artifact that is missing from the contract schema registry, when language/framework docs omit required matrix/checklist coverage, or when generated baselines drift from checked-in expectations.
• This is good for the project because it makes language support auditable and scalable. Adding or changing a language/framework now means updating a contract, matrix, schema, validator, and runnable evidence path together; it is much harder to accidentally ship an adapter that works for a happy-path demo but silently breaks graph identity, risk flows, framework virtual segments, or release evidence.
• We did this because PairOfCleats is only useful if cross-language indexing behaves predictably. USR gives the repo a stable substrate for future parser improvements, framework-specific extraction, risk modeling, retrieval ranking, and release readiness without every downstream consumer needing to know the quirks of every parser and framework compiler.

Production and release readiness

• Hardened npm run verify:production, release-check tooling, release-surface docs, readiness docs, and release evidence citations.
• Added checks that bind release metadata, trust material, and CI supplemental artifacts to the checked-out release SHA.
• Hardened release report validation for schema shape, ISO timestamps, status summaries, per-phase accounting, and TUI runtime-target metadata.
• Added SBOM/trust-manifest file existence validation and stricter supplemental artifact validation for CI summaries, coverage, trust status, and malformed JSON.
• Rejected unsafe release input/output shapes: repo-external paths, absolute/traversing archive entries, backslash entries, symlinked release-walk inputs, symlinked explicit release-check artifacts, symlinked readiness input roots, empty explicit release-check scope, and malformed surface archive manifests.
• Preserved exact TUI artifact download behavior and runtime target matching for release verification reports.
• Updated workflows/docs so release readiness gates are backed by current technical proof rather than stale process paperwork.

Shared-module ownership and facade reduction

• Continued the hard-cut shared-module deflation work so internal callers import narrow owners rather than broad root facades.
• Removed or deflated root shared modules after callers moved to narrower owners, including concurrency, subprocess, progress, runtime envelope, risk explain, cache, cache CAS, auto-policy, command registry, ONNX helper, CLI completion, progress-context, runtime-capability builder, env, artifact-IO, and related helper surfaces.
• Added migration recipes, boundary guardrails, cycle checks, and adoption tests so broad shared-module facades do not quietly re-enter internal code.
• Tightened retrieval CLI import-budget governance and lazy loading so the CLI graph stays below the current module ceiling.
• Split helper ownership by domain instead of preserving compatibility wrappers:
  • cache CAS helpers into focused path/metadata/object/gc/lease modules.
  • artifact IO helpers into constants/json/loaders/manifest/chunk-meta/optional-fallback owners.
  • env helpers into runtime/TUI/core ownership.
  • ONNX helpers into config/run-queue/tokenization modules.
  • CLI and TUI-specific helpers into their owning tool surfaces.

Duplicate-code reduction and jscpd governance

• Added the durable duplicate-audit setup:
  • jscpd@4.2.3
  • .jscpd.json
  • npm run audit:duplicates
  • docs/tooling/duplication-reduction-status.md
• Ran an intentional full duplicate audit and recorded the current checkpoint baseline.
• Used the saved jscpd report for follow-up reductions instead of repeatedly rerunning the audit.
• Added an exact-current saved-fragment refresh that scanned the saved 212-fragment report against current file contents and found 0 still-current fragments.
• Reduced duplicate families across production code, tooling, tests, benchmark helpers, retrieval fixtures, language adapters, SQLite fixtures, graph/context-pack rendering, risk output, LSP fixtures, VS Code runtime tests, editor packaging, VFS benches, USR bench/CI helpers, and many test harnesses.
• Documented the current duplicate-policy boundary: the 212-clone numeric audit is intentionally stale after follow-up reductions, and future duplicate work should start from a fresh intentional full audit or a fresh live duplicate signal.

Sweet16 and performance work

• Cleaned the Sweet16 CI performance batch so the latest benchmark JSON reports:
  • 14 ok
  • 0 errors
  • 0 timeouts
  • 0 skipped
  • no regression signals
• Tightened the benchmark runner report contract and JSON shape.
• Fixed report-contract handling for benchmark deltas and regression-signal interpretation.
• Optimized cache key builders and cache-hit contract paths.
• Reduced file metadata overhead with columnar optional-field handling and stable IDs.
• Improved SQLite build-from-artifacts behavior and bench contract coverage.
• Reduced artifact IO, file-meta, chunk-meta, relation build, repo-map, index-state, and minhash overhead where the branch had measured friction.
• Added or updated targeted perf/bench tests so the optimized paths remain guarded rather than only measured once.

Artifact, SQLite, cache, and indexing reliability

• Hardened artifact IO loaders, JSON/JSONL streaming, manifest handling, optional fallback behavior, byte guards, and small-shard parsing.
• Improved SQLite artifact-build and incremental-update paths, including chunk metadata streaming, meta fallback, gzip fallback, build-state handling, and benchmark contract tests.
• Refined cache contract surfaces, cache root handling, cache key generation, hit-rate contracts, and cache-migration coverage.
• Added guardrails around file metadata stable IDs, columnar roundtrips, membership performance, and streaming memory/reuse behavior.
• Continued Stage1 throughput and ordered commit-cursor work so contiguous windows, no-gap recovery, and release-facing Stage1 validations are present.
• Preserved risk interprocedural, snapshot/diff/as-of retrieval, relation boosts, ANN candidate safety, chargram, lexicon, and retrieval output surfaces in the release validation plan.

API, MCP, CLI, tooling, editor, and LSP surfaces

• Updated API and MCP contract docs/schemas and request projection helpers.
• Tightened MCP risk filter schemas, workspace selection schema literals, cleanup guards, and API/MCP test coverage.
• Improved CLI dispatch validation, wrapper flag alignment, service config coercion, and CLI completion rendering ownership.
• Hardened LSP/tooling provider behavior, including degraded preflight paths, workspace root selection, provider diagnostics, SourceKit/gopls handling, TypeScript/Rust/Dart fixture coverage, and provider process reuse tests.
• Updated VS Code and Sublime packaging/runtime surfaces, including shared editor package CLI behavior and VS Code runtime test helpers.
• Preserved TUI release/runtime behavior while moving supervisor-specific progress context to the tool surface that owns it.

Test runner, lane, and validation governance

• Updated tests/run.js and lane discovery so tracked .unit.js tests and runner selectors are discovered through the repo runner instead of requiring direct Node invocation.
• Added/updated generated lane manifests, test runner interface docs, contract matrix tests, docs freshness checks, markdown-link checks, and evidence-citation guards.
• Reworked test helper duplication into shared fixtures where it improved clarity and reduced repeated setup without hiding scenario-specific assertions.
• Split oversized readiness-gate tests into smaller focused files and shared fixtures to stay within repo file-size guidance.
• Kept repository test helpers responsible for PAIROFCLEATS_TESTING=1 setup rather than requiring ad hoc environment setup in every command.

Validation evidence

Recent local validation evidence on this branch includes:

• node tools\bench\bench-runner.js --suite sweet16-ci --json .testLogs\bench-sweet16.json --quiet
  • JSON: .testLogs\bench-sweet16.json
  • Result: 14 ok, 0 error, 0 timeout, 0 skipped, no regression signals.
• Focused post-performance validation run:
  • Log directory: .testLogs\run-1779479470090-ncqzlb
  • Covered cache contracts, SQLite artifact build contracts, file-meta roundtrip/perf/stable-ID paths, chargram guardrails/bench contracts, USR checklist docs, contract matrix docs, and markdown link checks.
• temp/validation/release-readiness-technical-validation-20260522.log
  • Release readiness gate focused selectors passed.
• temp/validation/readiness-evidence-citation-final-20260522.log
  • Readiness evidence and docs guard selectors passed.
• temp/validation/usr-current-evidence-handoff-validation-20260522.log
  • Current USR evidence handoff selectors passed.
• temp/validation/usr-full-conformance-20260522.json
  • Current full USR conformance artifact.
• docs/tooling/duplication-reduction-status.md
  • Documents the jscpd baseline, saved-report exact-current refresh, and current duplicate-work policy.
• docs/roadmap.md
  • Documents that current active roadmap initiatives are checkpoint clean or done, with future work gated by fresh evidence rather than stale plan documents.

Additional broad validation recorded by the roadmap includes production verification, release-lane docs/USR proof, Stage1/risk/snapshot proof, release runtime proof, full perf lane proof, shared-module governance proof, duplicate-code saved-fragment proof, and generated-surface freshness checks.

Current status after this branch

• No known active roadmap implementation batch remains open in the canonical roadmap.
• Duplicate-code work is checkpoint clean for the current saved-report refresh; future duplicate work should start with a new intentional npm run audit:duplicates pass or a fresh concrete duplicate signal.
• Sweet16 currently has no regression signals in the latest benchmark JSON.
• Production/readiness/USR/shared-module surfaces have current technical validation evidence recorded in durable docs and temp evidence artifacts.
• The main remaining branch-level work is integration against main: GitHub currently reports this PR as DIRTY, so merge conflicts need to be resolved before merge.